[CIEXE-2681] Use published Docker CI images with internal Buildx - #882

What does this PR do?

Replace all Docker 20.10/20.10-py3 CI image references with already-published, digest-pinned images containing internal Buildx v0.37.2-dd.3:

• Docker 27.3.1 for layer builds, layer-size checks, and layer bundles.
• Docker 29.4.0-noble-py3 for the existing Python-based integration/signing/publishing jobs and their utility jobs.

Both image manifests come from merged images/master commit af53db03938b864dfea79025c679312a0479726d (Docker build, Python Docker build). No new image publication is required.

Motivation

The existing Docker images report upstream Buildx v0.8.0 and miss our internal reliability patches. Reuse maintained images instead of creating additional Docker 20.10 variants in images#12452. Related: https://datadoghq.atlassian.net/browse/CIEXE-2681.

Noble's system Python is externally managed, and the published Python image lacks AWS CLI. Install awscli/python3-venv/python-is-python3 before the affected jobs, and create the Python virtual environment before installing dependencies. This preserves the required AWS commands without writing Python packages into the managed system interpreter.

Testing Guidelines

• Rendered the before/after pipelines with gomplate and parsed them with Ruby/Psych. Verified all 88 affected generated jobs change only image selection and required tool setup, and AWS/Python consumers receive that setup.
• Bash syntax and ShellCheck error checks pass. A local script harness creates a real virtual environment and confirms dependency installs use it (package downloads stubbed).
• Registry manifests and published provenance verified; both images package internal Buildx v0.37.2-dd.3.
• git diff --check passes.

Full container/CI integration validation remains. The local Docker filesystem is nearly full. The commit no longer contains a CI-skip directive. Normal pipelines include automatic sandbox publishing and integration deployments; a deployment-free validation run must set the existing SKIP_E2E_TESTS=true pipeline variable.

Additional Notes

This also upgrades Docker and the Python job OS from the legacy image; reviewers should validate representative layer builds and AWS/Python workflows before merge. Library code, Lambda runtime images, release conditions, and deployment targets are unchanged.

Types of Changes

• Misc (CI dependency upgrade)

Check all that apply

• This PR's description is comprehensive
• This PR passes the integration tests (requires an explicitly approved run)